Because of complexity of background replacement, the traditional method is inefficient and the accuracy is difficult to improve. To solve these problems, a fast image background replacement method based on dilated convolution, called FABRNet, was proposed. First of all, the first three parts of VGG (Visual Geometry Group network) model were used for convolution and pooling operations of input images. Secondly, the combination of multiple sets of dilated convolutions were embedded into convolution neural network to make the network have a large and fine enough receptive field; meanwhile, the residual network structure was used to ensure the accuracy of the information distribution in the convolution process. Finally, the image was scaled to the original size and output by bilinear interpolation algorithm. Compared with three classical methods such as KNN (K-Nearest Neighbors) matting, Portrait matting and Deep matting, the experimental results show that FABRNet can effectively complete the background automatic replacement, and has advantages in running speed.